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Preface 



In many real growing networks the mean number of connections per vertex increases with 
time. The Internet, the Word Wide Web, collaborations networks, and many others display this 
behavior. Such a growth can be called accelerated. We show that this acceleration influences 
distribution of connections and may determine the structure of a network. We discuss general 
consequences of the acceleration and demonstrate its features applying simple illustrating 
examples. In particular, we show that the accelerated growth fairly well explains the structure 
of the Word Web (the network of interacting words of human language). Also, we use the 
models of the accelerated growth of networks to describe a wealth condensation transition in 
evolving societies. 
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0.1 Acceleration 

The great majority of models of evolving networks contain a very important assumption. 
These models suppose that the total number of edges in a growing network is a linear function 
of its size, that is, of the total number of vertices. This linear growth does not change the 
average degree of the network ^ |[]. (Here, following standard terminology from graph 
theory, we call the total number of connections of a vertex its degree. Physicists often call this 
quantity "connectivity". The number of incoming edges of a vertex in directed networks is 
called in-degree, the number of outgoing edges is out-degree.) 

The first model for the growth of networks under mechanism of preferential linking, 
namely, the Barabasi-Albert model (see also Ref. is only one example of a linearly 
growing network from a very long list [|[ 0, ^ |[ [l^, [iTi |12|, [l3| ] . Thus, a linear type of growth 
is usually supposed to be a natural feature of growing networks. But let us ask ourselves, 
whether this very particular case, that is, the linear growth is so widespread in real networks. 
To answer this question we must look at existing empirical data. Let us start from the most 
well known nets. 

(i) The World Wide Web: 

Recall that the WWW is the array of its documents (pages) plus hyper-links, namely, 
mutual references in these documents. The WWW is a directed network. Although hyper- 
links are directed, pairs of counter-links, in principle, may produce undirected connections. 
Links inside pages (self -references) are usually not considered as edges of the WWW, so this 
network does not contain "tadpoles" (closed one-edge loops). 

According to Ref. in May of 1999, from the point of view of Altavista, the WWW 
consisted of 203 x 10^ vertices (URLs, i.e., pages) and 1466 x 10^ hyperlinks. The average 
in- and out- degree were fcj = ko ^ 7.22. The average in- and out-degrees are equal to 
each other, since all the connections are inside the WWW. (Notice that "physical" time is 
unimportant for us, so that, in principle, we might not mention any date.) In October of 1999 
there were already 271 x 10^ and 2130 x 10® hyperlinks. The average in- and out- degree 
have become ki = kg = 7.85. Thus, the average degree of the WWW is increasing. 

(ii) The Internet: 

Very roughly speaking, the Internet is a set of vertices, which are interconnected by wires. 
The vertices of the Internet are hosts (computers of users), servers (computers or programs 
providing a network service that also may be hosts), and routers that arrange traffic across the 
Internet. Connections are naturally undirected (an undirected network), and traffic (including 
its direction) changes all the time. Web documents are accessible through the Internet (wires 
and hardware), and this determines the relation between the Internet and the WWW. Routers 
are united in domains, however, this notion is not well defined for the Internet. In January of 
2001, the Internet contained already about 100 millions hosts. One should emphasize, that it 
is not the hosts that determine the structure of the Internet, but rather, routers and domains. In 
July of 2000, there were about 150 000 routers in the Internet JI^. Later, the number rose to 
228 265 (data from Ref. Thus, one can consider the topology of the Internet on a router 

level or inter-domain topology lO]. In the latter case, it is actually a small network. 

According to data of Ref. [n7t| for the inter-domain level of the Internet, in November of 
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1997 it consisted of 3015 vertices and 5156 edges, so that the average degree was k = 3.42. In 
April of 1998 there were 3530 vertices and 6432 edges, and the average degree was k = 3.65. 
In December of 1998 there were 4389 and 8256 edges, so the average degree was already 
equal to 3.76. Then, the average degree of the Internet on the inter-domain level is increasing. 

We have noted that domains in the Internet are poorly defined. Also, the last data of Ref. 
JTt] ] are for December of 1998. However, one may use more recent data on "autonomous sys- 
tems". Extensive data on connections of operating "autonomous systems" (AS) in the Internet 
are being collected by the National Laboratory for Applied Network Research (NLANR). For 
nearly each day, starting from November of 1997, NLANR has a map of connections of AS. 
These maps are closely related to the Internet graph on the inter-domain level. Statistical anal- 
ysis of these data was made in Ref. [[T^, p^. The data were averaged, and for 1997 the average 
degree 3.47 was obtained; in 1998, the average degree was 3.62, in 1999, k ~ 3.82. Again we 
see that the average degree of the Internet on the inter-domain level (more rigorously speak- 
ing, on the AS level) is increasing. One should add that the growth of the average degree of 
the net of AS was also indicated in Ref. [ |20| ] . 

Unfortunately, there are no reliable empirical data on the router level of the Internet to 
arrive at precise conclusions. In 1995, the Internet included 3888 routers with 5012 inter- 
connections 0, that is ~ 2.6. In 2000, there were ~ 150 000 routers and ^ 200 000 
interconnections between them, so that k ^ 2.7 [p3[]. These data are taken from different 
sources, they are not precise and cannot be compared. 

(iii) Networks of citations in scientific literature: 

Vertices of citation networks are scientific papers, directed edges are citations. One cannot 
update the list of references in a published paper, so that new edges do not emerge between 
old papers. The direction of an edge between two papers is rigorously determined by their 
ages, so that one may forget about the directedness of citation networks. Such citation graphs 
(see Fig. |l|) are actually very simple growing networks, and most of demonstrating models 
of growing networks belong to this class. Note that in electronic archives one can update old 
papers and lists of references in them. This produce new links between old papers, so that the 
networks of citations of electronic archives are not quite classical citation graphs. 




Figure 1: Scheme of the growth of a citation graph. New connections emerge only between a 
new vertex and old ones. New connections between old vertices are impossible. 

Statistics of citations in scientific journals was studied in Ref. [ pH (see the earlier empiri- 
cal study of the issue in Ref. p^]). These data were collected for a number of journals (about 
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10) in the period 1991-1999. In all the journals that were studied in Ref. [ pT| the average 
number of references in papers was found to increase. 

(iv) Collaboration networks: 

In the simplest version of a collaboration network, vertices are collaborators. A pair of 
vertices is connected together by an undirected edge if there was at least one act of collab- 
oration between them ^3|]. For example, in scientific collaboration networks (networks 



of coauthorships), vertices are authors, and edges are coauthorships [ |24| . Such networks are 
projections of more complex and informative bipartite graphs, which contains of two types of 
vertices: collaborators and acts of collaboration. Each collaborator is connected to all the acts 
of collaboration, in which he was involved. Empirical data are mostly collected for simple 
one-mode collaboration networks. 

Empirical data of Refs. [^s], Q for large scientific collaboration networks indicate the 
linear growth of their average degree with the increasing number of their vertices. This means 
that the total number of edges in a network increases as a square of the total number of vertices. 

Thus we see that the accelerated growth of networks is not an exception but rather a rule. 
On the contrary, the linear growth is a simple but very particular case. 



0.2 Reasons for the acceleration 

Why is the accelerated growth widespread? As an example consider the growth of the WWW. 
Let us discuss how new pages appear in the WWW (see Fig. ^ [H]. Discussion of the growth 
of the WWW may be found in Refs. [ ^7[ Suppose, you want to create your own personal 
home page. You prepare it, put references to some pages of the WWW (usually, there are 
several such references, but in principle the references may be absent), etc. But this is only 
the first step. You must make your page accessible in the WWW. Your system administrator 
puts a reference to it (usually one reference) in the home page of your institution, and your 
page in the Web. 




Figure 2: Scheme of the growth of the WWW. A new document of the WWW must have at 
least one incoming hyperlink to become accessible. It may contain any number of references 
to other pages of the WWW, but usually there are several such outgoing hyperlinks. Also, new 
hyperlinks emerge between old pages of the WWW. 

However, you proceed to work with your page. From time to time, you add new references 
to it. Of course, you may remove some old references, but usually the total number of refer- 
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ences in a page grows. Then the average degree of the WWW increases, that is, the growth of 
the WWW is naturally accelerated. 

0.3 Degree distributions of networks 
0.3.1 Types of degree distribution 

In this paper we restrict ourselves to degree distributions of networks. Most of empirical 
results are obtained for this simple basic characteristic. Unfortunately, a degree distribution 
(in-, out-degree distribution) is a restricted characteristic of networks. Indeed, degree is a one- 
vertex quantity, so that, in general, degree distribution does not yield information about the 
global topology of a network. 

In most of cases, for example, for growing networks, in which correlations between de- 
grees of vertices are strong [ pi] , |l8[[l^], a degree distribution is only the tip of the iceberg (see 
Fig. ^ a). Of course, if degree-degree correlations in a network are absent, then, knowing the 
degree distribution of a network, one can completely characterize the net (see Fig. ^ b). We 
face this situation in many equilibrium networks. 



Figure 3: Degree-degree correlations, which are necessary present in growing networks, make 
a degree distribution far less informative characteristic (a). The degree distribution of the equi- 
librium uncorrected network contains complete information about its structure (b). 




degree 
distribution 
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Furthermore, analytical results on percolation on networks [ p^ , pO| ], disease spread within 
them [|T], ^2|], etc. were obtained just for a simple construction without degree-degree corre- 
lations. This construction is a standard model of a maximally random graph with an arbitrary 
degree distribution taken from mathematical graph theory ("random graphs with restricted de- 
gree sequences") [Q. Luckily, it seems that main percolation and disease spread results that 
was obtained for equilibrium networks are still valid for non-equilibrium nets. 

What kinds of degree distributions are realized in networks? Here we list the main types 
with some simple examples of the corresponding networks. 

(a) Poisson degree distribution, P{k) = e^~^k /k\ (see Fig. ^ a). 

The Poisson distribution is realized in a classical random equilibrium graph of Erdos and 
Renyi in the limit of the infinite network, that is, when the total number of vertices 

TV is infinite. Pairs of randomly chosen vertices are connected by edges. Multiple edges 
("melons") are forbidden. One may create L edges in the graph, or connect pairs of vertices 
with the probability L/[N{N — l)/2]. In both these cases, the resulting graph is the same in 
the limit N ^ oo. 



log P(k) 



logP(k)* 




b) 



logk 



logk 




log P(k) 



e) 



logk 



Figure 4: "Zoology" of degree distributions in networks. Main types of a degree distribution 
in log-log plots. Poisson (a), exponential (b), power-law (c), multifractal (d), and discrete (e) 
distributions. 
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(b) Exponential degree distribution, P{k) ^ exp(— fc/const) (see Fig. ^ b). 

A citation graph (see Fig. with attachment of new vertices to randomly chosen old ones 
produces the exponential distribution, but this is only one possible example. (Let each new 
vertex have the same number of connections, that is, the growth is linear) 

Also, the exponential degree distribution is rather usual for many equilibrium networks 
that are constructed by mechanism of preferential linking. 

(c) Power-law degree distribution, P{k) ~ k ^ (see Fig. 0, c). 

Here the standard example is the Barabasi-Albert model [Q] (see also Ref. This grow- 
ing network is a linearly growing citation graph in that new vertices are attached to preferen- 
tially chosen old ones. "Popular" old vertices attract more new connections than "failures": 
''popularity is attractive". This is a quite general principle. For example, this one is incorpo- 
rated in the Simon model 0. In the Barabasi-Albert model, the probability that an edge 
becomes attached to some vertex is proportional to the degree k of this vertex. This yields 
7 = 3. If the probability is proportional to k + const (a linear preference function), 7 takes 
values between 2 and 00 as the constant changes from — 1 to 00 . 

Power-law distributions are usually called scale-free or fractal. 

(d) Multifractal degree distributions (see Fig. ^, c). 

This distribution has a continuum spectrum of power laws with different weights. The 
growth of a network may produce such a degree distribution if new vertices partially copy de- 
grees of old ones [^8[|. In particular, multifractal degree distributions emerge in some models 
of networks of protein-protein interactions [p9[]. Multifractal distributions is a more general 
case of a fat-tailed distribution than power-law distributions. Numerous empirical data were 
fitted by a power-law dependence. However, there were no attempts to check the possibility 
that at least some of empirical degree distribution are multifractal. 

(e) Discrete degree distributions (see Fig. |[ d). 

Deterministic growing graphs have a discrete spectrum of degrees. Recently, it was demon- 
strated that some simple rules of deterministic growth may produce discrete degree distribu- 
tions with a power-law decay [^^. Moreover, deterministic graphs from Refs. ^ 
have an average shortest-path length, which is proportional to the logarithm of their size. Fig- 
ure H shows a simple deterministic graph [^, ^ with the discrete degree distribution that is 
characterized by exponent 7 = 1 + In 3/ In 2. 

0.3.2 Power-law degree distribution 

Power-law (that is, "scale-free") degree distributions is a prominent particular case of fat- 
tailed degree distributions, which are widespread in real networks (both natural and artificial) 

||, Let us discuss briefly the general features of power-law distributions. 

One may ask, what are the possible values for 7? The first natural restriction follows from 
from the normalization condition / dkP{k) = 1 (in this discussion we change the correspond- 
ing sum to the integral). We may not be worried about the low-degree region, since the degree 
distribution is certainly restricted below some characteristic degree kg. Only the large degree 
behavior of the degree distribution is interesting for us. Therefore, the strong restriction is 
7 > 1, otherwise the integral is divergent. 

If a network grows linearly, so that the first moment of the distribution, that is, the aver- 
age degree k, is independent of time, then we have the second restriction / dkkP{k) < 00. 
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Figure 5: A simple deterministic graph ^jj] with a power-law discrete degree distribution. 
The growth starts from a single edge between two vertices. At each time step, each edge of 
the graph generates a new vertex, which became attached to both the end vertices of the mother 
edge. The average shortest-path length of this graph grows logarithmically with the total number 
of vertices. 

Therefore, 7 > 2 for linearly growing networks. 

Finite size effect cuts the power-law part of the degree distribution at large degrees. This 
produces size-dependent degree distributions. One may easily estimate the position of the 
cutoff kcut in the situation where 7 > 2. Let the total number of vertices in the net be t, 
and fco be some characteristic degree, below which the distribution is, for example, constant 
or even zero. Then, using the normalization / dkP{k) = 1 gives the power-law part of the 
degree distribution of the form P{k) ~ [(7— l)fcj~^]fc^''' for ko < k < kcut- 

When one measures the degree distribution of a network using only one realization of the 
growth process, strong fluctuations are observed at degree kf{t) that is determined by the 
condition tP{kf{t)) ^ 1. This means that only one vertex in the network has such degree. 
(More rigorously speaking, the number of such vertices is of the order of one.) This is the first 
natural scale of the degree distribution. 

One may improve the statistics by measuring many realizations of the growth process, or, 
for example, by passing to the cumulative distribution Pcnm = Jj^ dk P{k). Both these tricks 
allow us to reduce the above fluctuations. However, we still cannot surpass the next threshold 
that is originated from the second natural scale, kcut'- tPcum{kcut{t)) ^ 1. This means that 
only one vertex in the network is of degree greater than kcut- (Again, more rigorously, the 
number of such vertices is of the order of one.) Using the above expression for P{k) gives 



kcut-kof^^-^-'K (1) 

Notice that the only reason for this estimate for the cutoff is the natural scale of the prob- 
lem. Hence more convincing arguments are necessary. The estimate was checked for some 
specific models. A growing network Jl3[ ] was solved exactly, and the exact position of the 
cutoff have coincided with Eq. (jl|). The degree distribution of this network has a typical form 
(see Fig. Notice a hump near kcut in Fig. ^ This is a trace of initial conditions. Simula- 
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tion of a scale-free equilibrium network [ p3| ] also yielded the cutoff at this point. However, the 
introduction of the death of vertices in the network may change the estimate (|l]). This factor 
also removes the hump from the degree distribution. Here we do not consider such situations. 



log p(ky 




Figure 6: The typical form of a power-law degree distribution of finite growing networks. The 
finite-size cutoff is given by Eq. (|^). The hump near the cutoff depends on initial conditions (we 
do not account for the factor of mortality). 

The cutoff (jl]) hinders measurements of power-law dependences in networks [|r3|l . From 
Eq. (jl]) one sees that the measurements of large enough 7 are actually impossible. Indeed, 
in this case kcut is small even for very large networks, and there is no room In fco < In fc < 
In kcut for fitting. 

What is the nature of power-laws in networks? One may directly relate them to self- 
organized criticality. While growing under mechanism of preferential linking, networks self- 
organize into scale-free structures, that is, are in a critical state. This critical state is realized for 
a wide range of parameters of preferential linking, namely for any linear preference function 
(more rigorously, for any preference function which is asymptotically linear at large k ^). 
The linear growth of networks may produce scale-free structures. Then, one may ask: What 
degree distributions does the accelerated growth produce? 



0.4 General relations for the accelerated growth 

Let us start with general considerations and do not restrict ourselves by some specific model. 
Let the average degree grows as a power of i, fc cx t°-, that is, the total number of edges 
L{t) (X t"'~^^. Here a > is the growth exponent. The consideration is valid not only for 
degree, but also for in-, and out-degrees, so we use the same notation k for all them. The 
power-law type of acceleration we have chosen since one may hope that it provide scale- 
free networks. We suppose from the very beginning that this is the case and then check our 
assumption. 
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For the accelerated growth, the degree distribution may be non-stationary. It is natural to 
choose its power-law part in the form 



P{k,t) - t'k- 



(2) 



Here we have introduced new exponent z > ^ ^ (recall that we consider only a > 0). 
This form is valid only in the range fco(<) < k < kcut{t)- Using the normalization condition 

/m*)'^''*"^"^ ~ 1 gives 

ko{t) - . (3) 

This estimate is valid for any 7 > 1. 

The cutoff kcut{t) is estimated from the condition t ^.^^ dk t^k~^ ^ 1. Therefore, 



(4) 



(compare with Eq. (|l|) for the linear growth.) Equation (^ holds for any 7 > 1. 

We will consider two cases (see Fig. 0), 1 < 7 < 2 and 7 > 2. Recall that we do not 
account for mortality of vertices. 



log P(k)' 




Figure 7: Schematic plot of a time-dependent degree distribution of networks that grow in the 
accelerated mode. Arrows show how the degree distribution changes with time. 



(i) 1< 7 < 2. 

Recall that the average degree distribution k{t) ^ t"^. Then 



Here the value of the integral is determined by its upper limit. Therefore, (z + l)/(7 — 1) 
a + 1, and the cutoff is of the order of the total number of edges in the network. 
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K^tit) ^ ^ L{t) . (5) 

But this is the maximum possible degree in the problem. In this sense, any cutoff of a degree 
distribution is absent if 7 < 2. From the last relation, we obtain 7 exponent in such a situation. 



7 = 1 + ^. (6) 

Here,for7 < 2, one assumes that z < a. The lower boundary for 7, namely 7 = l + l/(a + l) 
is approached when z = 0, that is, when the distribution is stationary. 

(ii) 7 > 2. 

The integral for the average degree is determined by its lower limit 
Hence 



7 = 1 + - (7) 
a 

and z > a to keep 7 > 2. Notice that this relation is not valid for a = 0. One sees that, in this 
case, the degree distribution cannot be stationary: z > a > 0. 

0.5 Scaling relations for accelerated growth 

For simple scale-free networks that grow in a linear mode, simple scaling relations can be 
written [0 ^. Let us briefly describe the corresponding scaling relations for the accelerated 
growth. If vertices in a growing network do not die, one can label them by their "birth date" 
< s < t. We denote by p{k, s, t) the probability that the vertex s is of the degree q. The 
average degree of a vertex s at time t is fc(s, t) = f dk kp{k, s, t). 
For networks that we consider the fc(s, t) is 

I(s,^)cx^^(0"^ (8) 



where [3 and 7 are scaling exponents. One can show [ J45| ] thatp(fc, s, t) — [l/fc(s, t)]g[k/k{s, <)] 
where <?[ ] is some scaling function, therefore 



p(k,s,t)^t-'[^%\kt-'(^^ 



(9) 
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Using the relation P{k,t) — t ^ /J dsp{k, s, t) yields 

) 

dx t-^ g[kt-^ xf^] cx t^lf^k-^^^"^ cx t''k-'< , (10) 
whence we obtain relations for the scaling exponents: 

7=1 + 1//? (11) 

and 

z = 5/l3. (12) 
Accounting these relations gives the scaling form 



sl/(7-l) 



k- ' 



't(^ + l)/(7-l) 

Similarly, one can find the scaling form for the degree distribution. 



(13) 



P(fc, t) = ek-''G{kt-'^^+'-^'^) = i^fc^^G(fct-(i+^)/('^-i)) , (14) 

where G'( ) is a scaling function. When z = 0, Eqs. ( p3[ ) and (|l^ coincide with the scaling 
relations ^ for linearly growing networks. 

Notice that it is sufficient to know a and only one exponent of 7, /?, z, (5, or x to find all the 
others. 



0.6 Degree distributions produced by the acceleration 

Let us discuss several illustrative examples. To begin with, we consider a network growing 
under mechanism of preferential linking, in which number of new connections increases as a 
power law in time. At this point we do not discuss the origin of this power-law dependence. 
Let it be equal to cot°, where cq is some positive constant. Here is convenient to study the 
in-degree distribution, so that k will be in-degree. In such an event we are interested only in 
incoming connections, so that the outgoing ends of new edges may be attached to any vertices 
of the network or even be outside of the net. 

Let the probability that a new edge becomes attached to a vertex of in-degree k be propor- 
tional to fc -I- A{t), where A{t) is some additional attractiveness of vertices. Two particular 
cases of this linear preferential linking are considered below in the framework of a simple 
continuum approach |^ ^ p3[ ]. 
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0.6.1 Model for 7 < 2 

If the additional attractiveness is constant, A ~ const, the continuum equation for the average 
in-degree k{s,t) of individual vertices that born at time s and are observed at time t is of the 
form 



dk{s,t) _ „ k{s,t)+A 

r — Cnt — 7 — 

dt J^du[k{u,t) + A] 



Cot" , (15) 



with additional starting and boundary conditions fc(0, 0) = and k{t,t) = 0. Here we 
supposed that new vertices have no incoming edges. We use this assumption only for brevity. 
Naturally, the total in-degree of the network is f * duk{u, t) — cot'^+^/ (a + 1). This also can 
be seen by integrating both the sides of Eq. ( p5[ ) over s. Accounting the last equality yields 
the solution of Eq. (|l5|): 



-(1+1) 



k{s,t)^A[-) . (16) 



Therefore, /3 exponent equals a + 1 > 1, so that using scaling relation ( |1 1[ ) gives 

7 = 1 + < 2 . (17) 

a + 1 

One may also apply the following simple relation of the continuum approach: 



P(fc,t) = - f ds6{k-k{s,t))^-- (^^^'^^^ 
t Jq t 



ds 



(18) 



s—k{s,t) 



This equality follows from the fact that the solution of the master equation for the probabiUty 
p(fc, s, t) in the continuum approximation is the (5-function. From Eqs. (|l6|) and (|l7|) we obtain 
the in-degree distribution 

41/(1+1) 

P(fc, t) = ^.-[1+1/(1+1)1 ^ (19) 

a + I 



which is stationary. We have shown in Sec. 0.4 that when 7 = l + l/(a+l), the (in-) degree 



distribution must be stationary, and exponent z is zero. This is the case for the network under 
consideration. 



0.6.2 Model for 7 > 2 



Now we choose a different rule of attachment of new edges to vertices. Let the additional 
attractiveness be time dependent. Furthermore, let it be proportional to the average in-degree 



16 



of the network, cot°-/{a + 1), at the birth of an edge, A{t) = Bcof^/ia + 1)). Here B > is 
some constant. Analogously to the above we obtain the non-stationary in-degree distribution 



P{k,t) - ta(l+B)/(l-Ba)^-[l + (l + B)/(l-Ba)] ^20) 

for k:^ t"". Hence 7 exponent is 



l + B 

7-1 + 1 7^>2. (21) 

1 — Ba 

The scaling regime is reaUzed when Ba < 1. 



0.6.3 Dynamically induced accelerated growth 

We have shown above that the power-law growth of the total number of edges in a network 
(or its average degree) produces fat-tailed distributions. Now we discuss reasons the for the 
power-law growth. 

Consider an undirected citation graph, in which each new vertex becomes attached to a 
randomly chosen old one plus to some of its nearest neighbors, to each one of them with 
probability p (see fig. For the total number of edges L{t) one can write 



L{t + 1) - L{t) = 1 + pk{t) . (22) 
Here we use the continuum approximation. k{t) — 2L{t) /t, therefore 



l±[tk{t)] = \+pk{t). (23) 

Forp < 1/2, the solution of this equation approaches the stationary limit k = 2/(1 — 2p) 
ast ^ 00. In this case the degree distribution is stationary, and 7 exponent is 7 = 1 + 1/p > 3. 

The situation for p > 1/2 is quite different, the average degree of the network growth 
as a power law, k{t) ^ t^^"^ for large networks. This produces non-stationary distribution 
P{k) cx i^fc-^ with 7 = 1 + 1/(1 - p) > 3 and z 1/(1 - p) - 2. Of course, other 
mechanisms for the accelerated growth are also possible. 



0.6.4 Partial copying of edges and multifractality 

From Eq. (jlj) for the cutoff of a power-law (or, which is the same, fractal) distribution, one 
sees that the size dependence of the moments M„j(t) = J dk k™P{k, t) of this distribution is 



(24) 



0.7 Evolution of the Word Web 



17 




Figure 8: One of possibilities to obtain the acceleration. In this citation graph, each new vertex 
becomes attached to a randomly chosen old one plus to some of its nearest neighbors. 



where T(m) exponent is a linear function of the order m of a moment, T(m) = (m — l)/(7 — 
1) — (7 — 2)/(7 — 1). Just the linearity of T(m) defines a fractal distribution. The size 
dependence of the moments of a multifractal distribution also has the form (p4|), but its T(m) 
exponent is a nonlinear function of m. 

Multifractal distributions are a more gen eral c ase of fat-tailed distributions than a power- 
law (fractal, scale-free) dependence. In Sec. 3.6.3 we have shown how the accelerated growth 
may generate fractal distributions. However, this is only a particular possibility. Partial copy- 
ing (partial inheritance) of degrees of old vertices by newborn ones together with the pref- 
erential attachment of some extra new edges usually provides networks, which grows in a 
nonlinear way and have multifractal degree distributions. 

A simple consideration of this problem can be found in Ref. [|3^. Note that the acceler- 
ation and the multifractality of the degree distribution were obtained in a similar model 
for protein-protein interaction networks. In this model, duplication of vertices with edges 
attached to them and breaking of some connections of parent vertices were used instead of 
partial copying in Refs. 



0.7 Evolution of the Word Web 

The weak point of network science is the absence of a convincing comparison of numerous 
schematic models with real networks. Most of models of growing networks only demonstrate 
intriguing effects but, in fact, are very var from reality. Available empirical data usually can 
be explained by applying various models with fitting parameters. As a rule, only the exponent 
of the empirical degree distribution is used for comparison. 

Here we consider an exceptional situation, where a reasonable comparison of the model 
of a growing network with empirical data is possible without any fitting. Moreover, it is the 
idea of the accelerated growth that yields an excellent agreement. 

The problem of human language is a matter of immense interest of various sciences. How 
did language begin? How does language evolve? What is its structure? Quite recently, a novel 
approach to language was proposed [Q. Human language was considered as a complex 
network of interacting words. Vertices in this Word Web are distinct words of language, and 
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undirected edges are connections between interacting words. 

Words interact when they meet in sentences. Different reasonable definitions yield very 
similar structures of the Word Web. For example, we can connect the nearest neighbors in 
sentences. This means that the edge between two words of language exists if these words are 
the nearest neighbors in at least one sentence in the bank of language. One sees that multiple 
connections are absent. Of course, this is a rather naive definition, but it is also possible 
to account for other types of correlations between words in a sentence [Q. The resulting 
network gives the image of language, which is available for statistical analysis. 

The empirical degree distribution of the Word Web is very complex (see Fig. H). 
Therefore, a perfect description of these data without fitting would be convincing. Indeed, it is 
hardly possible to describe such a complex form of the distribution completely by coincidence. 
We show below that a minimal model of the evolving Word Web |p7[|, with only known 
parameters of this network, provides such a perfect description. 




Numbers of connections of words, k 



Figure 9: Empirical degree distribution of the Word Web (points) ||4q]. Empty and filled circles 
correspond to different definitions of the interactions between words in sentences. The solid line 
shows the result of our calculations using the known parameters of the Word Web, namely 
the size t ~ 470 000 and the average number of connections, k{t) ~ 72. The arrows indicate 
the theoretically obtained point of crossover, kcross, between the regions with exponents 3/2 
and 3, and the cutoff kcut of the power-law dependence due to finite-size effect. 

In Ref. [p^, the Word Web was constructed after processing 3/4 million words of the 
British National Corpus. The British Corpus is a collection of text samples of both spoken 
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and written modern British English. The resulting network contains about 470 000 vertices. 
The average degree is A: w 72. These are the only parameters of the network we know and can 
use in the model. 

Notice that the quality of the empirical data is is high: the range of degrees is five 
decades. The empirical degree distribution has two power-law regions with exponents 1.5 and 
about 3 (the latter value is less precise, since statistics in this region is worse). The crossover 
point and the cutoff due to finite-size effect can be easily indicated (see Fig. 

We treat language as a growing network of interacting words. At its birth, a new word 
already interacts with several old ones. New interactions between old words emerge from time 
to time, and new edges emerge. All the time a word lives, it enters in new "collaborations". 
Therefore the number of connections grows more rapidly than the number of words: the 
growth of the Word Web is accelerated. 

How do words find their collaborators in language? Here we again use the idea of prefer- 
ential linking [||, again the principle "popularity is attractive" works. 




Figure 10: Scheme of the Word Web growth. At each time step a new word emerge, so that 
t is the total number of words. It connects to m 1 preferentially chosen old words. Si- 
multaneously cmt new edges emerge between pairs of preferentially chosen old words. We 
use the simplest rule of the preferential attachment when a node is chosen with the probability 
proportional to the number of its connections. 

We use the following rules of the network growth (see Fig. |l^ [^^. 

(1) At each time step, a new vertex (word) is added to the network, and the total number 
of words is t. 

(2) At its birth, a new word connects to several old ons. Let, in average, this number be m, 
so that this number is not necessary integer. We use the simplest natural version of preferential 
linking: a new word become connected with some old one i with the probability proportional 
to its degree ki, like in the Barabasi- Albert model 

(3) In addition, cmt new edges emerge between old words, where c is a constant coefficient 
that characterizes a particular network. If each vertex makes new connections with a constant 
rate, this linear dependence on time naturally arises. These new edges emerge between old 
words i and j with the probability proportional to the product of their degrees kikj []lO|]. 

These simple rules define the minimal model that can be solved exactly. Here we discuss 
only the results of the continuum approach. In this case, the approach gives an excellent 
description of the degree distribution and the proper values of exponents. 
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In the model that we discuss, words are actually considered as collaborators in language. 
In our approach the essence of the evolution of language is the evolution of collaborations 
between words. Therefore the situation for the Word Web should be rather similar to that for 
networks of collaborations. The equivalent model was applied to scientific collaboration nets 
[p^, but the more complex nature of these networks makes the comparison impossible. 

As above, in the continuum approximation, we can write the equation for the average 
degree at time t of the word that emerged at time s: 



dt J^dukiu,t) 



(m + 2cmt) yj./ , (25) 



where the initial condition is k{0, 0) = and the boundary one is k{t, t) = m. 

One can see that the total degree of the network is du k{u, t) = 2mt + cmt^, so that its 
average degree at time t is equal to k{t) = 2m + cmt. 

The solution of Eq. is of a singular form 

k{s,t) = m( -— . (26) 

\ cms J \ 2m + cms / 

The form of this equation indicates the presence of two distinct regimes in this problem. Using 
Eqs. (|l8|) and ( ^6| ) readily yields the non- stationary degree distribution 

P,M) = l£;e±££>i, (27, 

ct I + cs k 

where s = s{k, t) is the solution of Eq. (|6|). Notice that, formally speaking, the number m is 
absent in Eq. ( |2^. T his is the consequence of our definition of the coefficient cm (see above). 

From Eqs. ( [26| ) and (p7[), one sees that the non-stationary degree distribution has two 
regions with different behaviors separated by the crossover point 

kcross ~ mV^t{2 + ctfl'^ . (28) 

The crossover moves in the direction of large degrees as the network grows. Below this point, 
the degree distribution is stationary. 



Above the crossover point, we obtain the behavior 

P(fc,i)^(?!IL±£!^fc-3. (30) 
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so that the degree distribution is non-stationary in this region. Thus, we have obtain two 
distinct values for the degree distribution exponent, namely, 3/2 and 3. 

The model that we consider has two limiting cases. When c = 0, it turns to be the 
Barabasi-Albert model, where 7 = 3. When m is small but cm is large, we come to the 
network from Sec. 3.6.1 which has 7 = 3/2 and a stationary degree distribution. Thus these 
two values of 7 are not surprising. The important point is that the crossover is observable even 
though cmt ^ m. 

The degree distribution has one more important point, the cutoff produced by finite-size 
effe ct. We estimate its position from the condition t dkP{k, t) ^ 1 (see Sees. 13 and 
14|). This yields 



fccut - y ^(2m + cmt)3/2. (31) 
Using Eqs. (El) and (53) one can estimate the number of words above the crossover: 



POO 

I 777 

N,^t dkP{k, t)^—. (32) 

We know only two parameters of the Word Web that was constructed in Ref. [p^, namely 
t = 0.470 X 10® and k{t) = 72 = 2m + cmt « cmt. About m we know only that it is of 
the order of 1. From the above relations, one sees that the dependence on m is actually weak 
and is not noticeable in log-log-scale plots. In fact, m is inessential parameter of the model. 
Hence we can set its value to 1. 

In Fig. ^, we plot the degree distribution of the model (the solid line). To obtain the 
theoretical curve, we used Eqs. ( |2^ and ( p7[ ) with m = 1 and c « k{t)/t. A rather inessential 
deviations from the continuum approximation are accounted for in the small-degree region 
(fc ~ 10). One sees that the agreement with the empirical data is fairly good. Note that 
we do not used any fitting. However, for a better comparison, in Fig. |[ the theoretical curve 
is displaced upwards. Actually, this is not a fitting, since we have to exclude two empirical 
points with the smallest degrees. These points are dependent on the method of the construction 
of the Word Web, on specific grammar, so that any comparison in this region is meaningless 
in principle. 

From Eqs. ( ^8| ) and (|3l|), we find the characteristic values for the crossover and cutoff, 
kcross ~ 5.1 X 10'^, that is, logj^Q kcross ~ 3.7, and log^Q fccut ~ 5.2. From Fig. ^ we see 
that these values coincide with the experimental ones. We should emphasize that the extent 
of agreement is truly surprising. The minimal model does not account for numerous, at first 
sight, important factors, e.g., the death of words, the variations of words during the evolution 
of language, etc. 

The agreement is convincing since it is approached over the whole range of values of k, 
that is, over five decades. In fact, the Word Web turns out to be very convenient in this respect 
since the total number of edges in it is extremely high, about 3.4 x 10^ edges, and the value 
of the cutoff degree is large. 

Note that few words are in the region above the crossover point kcross ~ 5.1 x 10'^. These 
words have a different structure of connections than words from the rest part of language. 
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With the growth of language, kcross increases rapidly but, as it follows from Eq. (^2|), the 
total number of words of degree greater than kcross does not change. It is a constant of 
the order of m^/ {8cm) ^ l/(8c) « t/ (8fc) ^ lO'^, that is of the order of the size of a small 
set of words forming the kernel lexicon of the British English which was estimated as 5, 000 
words | p8| ] and is the most important core part of language. Therefore, our concept suggests 
that the number of words in this part of language does not depend essentially of the size of 
language. Formally speaking, the size of this core determined by the value of the average rate 
c with which words find new partners in language. 

If our simple theory of the evolution of language is reasonable then the sizes of the cores 
of primitive languages are close to those for modern "developed" languages. 

0.8 Wealth distribution in evolving societies 

Ideas from network science can be applied to various problems. Here we show how the idea 
of the nonlinear growth works in econophysics. 

One of the basic problems of econophysics is wealth distribution. Usually, wealth distribu- 
tion is treated by using so called stochastic multiplicative models. The standard description of 
these stochastic multiplicative processes is provided by the generalized Lotka-Volterra equa- 
tion . The preferential linking mechanism, that is, the general ''popularity is attractive" 
principle, provides the stochastic multiplicative dynamics of networks [g]. Therefore, results 
that were obtained for networks may be easily interpreted in terms of wealth distribution. 

Let us discuss briefly wealth distribution in stable (stagnating), developing, and degrading 
(dying) societies. For simplicity, in our very schematic consideration, we do not account for 
mortality, redistribution and loss of money, inflation, and many other important factors. Let 
there be one birth per time step. Therefore, there are t members of the society at time t. Thus 
we consider growing (non-equilibrium) societies. 

In stable societies, wealth per member (the average capital, the average amount of money) 
does not change with time, and the input flow of capital is constant. In developing societies, 
the average wealth and the input flow of capital grow with time. In degrading societies, these 
quantities decrease. 

One introduces the distribution function of wealth, P{k, t). If this distribution is a power 
law, P{k) ^ k~^ , and 7 < 2, the society is "unfair": few persons keep a finite fraction of the 
total wealth. If 7 > 2, the society is "fair" . The wealth condensation transition [ ]5l| ] occurs 
when P(fc) passes over the dependence. When P{k) decreases more rapidly than a power 
law, e.g., the function is exponential, the society is " superfair" . 

To study wealth distribution in various societies, we consider the simplest demonstrating 
case of a power- law input flow of capital t". Growth exponent a indicates the type of society, 
a = corresponds to stable societies. Positive and negative a exponents provide developing 
and degrading societies, respectively. 

Let us discuss the simplest situation. We assume that money attract money. While trying to 
diminish inequality, society permanently distribute some fraction of wealth "fairly" (equally) 
among its members. Another way to make life better for all is to provide everybody with a 
starting capital. Society also provides its members by the educational etc. "capital" which 
can also attract money. Such a factor, additional attractiveness. A, also proportional to the 
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average wealth. It may be provided only once, at the birth, A{s) (s < t is the birth time of an 
individum), it may increase equally for all persons, A{t) (t is the age of a society), but in both 
cases the effect is qualitatively similar to starting capital. We consider the first possibility, that 
is, the providing with some starting capital at the birth as the simplest. 

We again apply the continuum approach. Then fc(s, t) is the average wealth of the person 
that was born at time s < t,t is the present time. 



0.8.1 Stable (stagnating) societies 

Let rris be starting capital and m extra wealth be distributed at each time step. A is a constant 
additional attractiveness. The total input flow of wealth is equal to m + TOj. A fraction p 
of the flow m is distributed among members of the society randomly, that is, "fairly", the 
flow (1 — p)m is distributed preferentially with probability proportional to your wealth. The 
continuum approach equation for the average individual wealth k{s,t) is of the form 



dk{s,t) pm 
dt ^ T 



+ (1 — p)ni 



k{s,t)+A 
/„* du [k{u, t) + A] 



(33) 



with the initial condition /c(0, 0) and the boundary one /c(t, t) — rag. Integrating (by 
parts) both the sides ofEq . (^3^ over s yields naturally ds k{s, t) = (m + ms)t. Similarly 
to the calculations of Sec. 3.6. l|, we obtain the power-law wealth distribution with 7 exponent 



7 = 2 + ^^" + "^; + -^ >2. (34) 

(1 — pjm 

Thus, in stable societies, 7 > 2, so that a stagnating society is fair. 



0.8.2 Developing and degrading societies 

Here we discuss a natural case: let your starting capital be proportional to the average wealth 
in the society at your birth, ras{t) ~ dmt", where c? is a positive constant. In addition, the 
wealth mt" is distributed among members of the society at each increment of time. The wealth 
prnf^ is distributed equally. The wealth (1 — p)mt" is distributed preferentially (money come 
to money). For brevity, we set A{s, t) — 0. Then we have 



dkis^ ^ ^^^p_ 
dt t 



(1 - p)mr 



k{s,t) 
lo du (u, t) 



(35) 



The initial and boundary conditions are fc(0, 0) = and k{t, t) = dmt°', respectively. From 
Eq. @, one sees that J^dsk{s,t) = m{l + d)r+7(a + !)■ 

From Eq. (35) we obtain the wealth distribution for various values of the parameters of 
the problem, p, d, and a. 
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(i) When a > (1 — p)/(p + d), or, in other words, p > (1 — ad) /{I + a), the wealth 
distribution is exponential (the "superfair society"). 

(ii) For a < (1 — p)/ {p + d), we obtain the power-law wealth distribution with exponent 



, = 2 + -(i±iM^. ,36) 

1 — p — a[p + d) 

One sees that 7 = 2 at a = —1. This corresponds to the "wealth condensation transition" 
from the "fair" society (7 > 2 for a > —1) to the "unfair" one (7 < 2 for a < —1). The 
resulting phase diagram is shown in Fig. [ri|. Note that the position of the wealth condensation 
transition does not depend on particular values of p and d. Therefore, even if a significant part 
of new wealth is distributed equally, rapidly degrading societies are necessarily unfair! 

The general picture of wealth distribution in our minimal approach is quite natural. Ex- 
tremely degrading societies are unfair It is impossible to approach any "fairness" by the "fair" 
distribution of any part of new wealth in such a situation. "Fair" societies are possible only 
if there is some progress or the degradation is rather modest. Only in fair societies "fair" 
distribution of new wealth produces visible results. 



CONCLUSIONS 

The nonlinear growth of networks is more general situation than the linear growth. In real 
evolving networks, the nonlinear, in particular, accelerated growth is widespread and is the 
rule and not the exception. In many cases, it is impossible to understand the nature of an 
evolving network without accounting for this acceleration. 

The complicating circumstance is that existing empirical data clearly indicate the presence 
of the acceleration but usually fail to yield its quantitative description. Theoreticians may eas- 
ily choose any functional form for the non-linear growth, but do these beautiful dependences 
have any relation to reality? 
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Figure 11: Phase diagram of evolving societies. At each time step, the wealth pmt°' is dis- 
tributed equally between all members of a society, the wealth (1 — p)mt°' is distributed pref- 
erentially (money come to money). The individual starting capital is dmt°' . In our schematic 
model, the wealth distribution of the "superfair" society is exponential. The wealth distribution 
of the fair society is a power-law with exponent 7 > 2. For the unfair society 7 < 2. 
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